% !TEX root = EUDAQUserManual.tex
\section{ROOT TTree Converter}
\label{sec:TTreeConverter}
%\author{M. Sohail \textsc{Amjad} \footnote{ \href{mailto:s.amjad@ucl.ac.uk} {s.amjad@ucl.ac.uk} } }

This Annexe\footnote{\href{mailto:s.amjad@ucl.ac.uk} {Authored by Sohail Amjad(s.amjad@ucl.ac.uk)}} provides details of a working example of a converter from \lstinline[style=cpp]{raw} format to \lstinline[style=cpp]{ROOT TTree} format mentioned in \ref{sec:RawEvent2TTreeEventConverter}. Any issues encountered during implementation of the instructions in this document should be reported at the dedicated platform.\footnote{\href{https://github.com/eudaq/eudaq/issues}\texttt{https://github.com/eudaq/eudaq/issues}}

\subsection{Event Structure in EUDAQ} 

In the EUDAQ data acquisition framework,\lstinline[style=cpp]{eudaq::Event} is the most important object in terms of data handling. It stores the data generated by the device under test (DUT). It is created by the \lstinline[style=cpp]{Producer} component of EUDAQ. The Producer then feeds physics data to fill it. It is the base class in EUDAQ and is serializable. Any object of type \lstinline[style=cpp]{Event} for any linked processes including Data Collector and Data Converter are to be derived from this class. The member variables of \lstinline[style=cpp]{eudaq::Event} are listed in table \ref{tab:eventdata2}. 

\begin{table}[h]
\centering
\small
\begin{tabular}{ l | l | l }
  \textbf{variable} & \textbf{C++ type} & \textbf{Description}\\
  \hline
  \texttt{m\_type} & \texttt{uint32\_t} & event type\\
  \texttt{m\_version} & \texttt{uint32\_t} & version\\
  \texttt{m\_flags} & \texttt{uint32\_t} & flags\\
  \texttt{m\_stm\_n} & \texttt{uint32\_t} & device/stream number\\
  \texttt{m\_run\_n} & \texttt{uint32\_t} & run number\\
  \texttt{m\_ev\_n} & \texttt{uint32\_t} & event number\\
  \texttt{m\_tg\_n} & \texttt{uint32\_t} & trigger number\\
  \texttt{m\_extend} & \texttt{uint32\_t} & reserved word\\
  \texttt{m\_ts\_begin} & \texttt{uint64\_t} & timestamp at the begin of event\\
  \texttt{m\_ts\_end} & \texttt{uint64\_t} & timestamp at the end of event\\
  \texttt{m\_dspt} & \texttt{std::string} & description\\
  \texttt{m\_tags} & \texttt{std::map<std::string, std::string>} & tags\\
  \texttt{m\_blocks} & \texttt{std::map<uint32\_t, std::vector<uint8\_t>>} & blocks of raw data\\
  \texttt{m\_sub\_events} & \texttt{std::vector<EventSPC>} & pointers of sub events\\
\end{tabular}
\caption{Variables of eudaq::Event.}
\label{tab:eventdata2}
\end{table}

%\includegraphics[width=0.99\textwidth, height=6cm]{eudaqevent.pdf}

Each variable describes a certain parameter/characteristic in the \lstinline[style=cpp]{Event} data. Some parameters are common and could be termed as $basic$ parameters such as \lstinline[style=cpp]{m_run_n, m_ev_n, m_tg_n,} etc which represent Run number, Event number and Trigger number respectively. As is evident from the naming scheme, the \lstinline[style=cpp]{m_run_n} represents the number of run which could be same for multiple events while the \lstinline[style=cpp]{m_ev_n} parameter should vary from event to event. All of these mentioned parameters are of type \lstinline[style=cpp]{unint_32} i.e. unsigned integer with 32 bit. Other parameters which contain the hardware specific data can vary from device to device in type and size. The \lstinline[style=cpp]{m_block} variable in the above list holds the physics data from hardware. The content and type of these data is known to the user only. As is evident from their types, these can be more complex objects such as vectors and maps. %The dedicated conversion of this part of the data is left to the user. 

An \lstinline[style=cpp]{Event} may also contain sub-structure depending on the circumstances. These can vary in number and size significantly. The variable \lstinline[style=cpp]{m_extend} is used to identify and point to such sub-events. A complete and adequate handling of the \lstinline[style=cpp]{Event} needs taking care of these sub-blocks. 

%\subsection{Data Conversion in EUDAQ }EUDAQ provides the support to convert data from the native {\texttt raw} format to other formats. Two data converters are already available and can be built during the compilation of EUDAQ by providing adequate dependencies.  \subsubsection{ Raw Event to Standard Event Converter}  A conversion process which converts the raw data to a standard format for pixel type detectors. It is a representation of a tracking-hit event in the sensor planes of the beam Telescope. Each sensor plane is treated as a pixel detector. The Event can be decomposed in {\texttt x\_pixel} and {\texttt y\_pixel} objects and those are saved in the {\texttt eudaq::StandardPlane}. The conversion iterates over all the hits found in the {\texttt eudaq::rawEvent}.   \subsubsection { Raw Event to LCEvent Converter} Another converter supported in EUDAQ is the {\texttt LCEvent} Converter. It takes the member variables of the raw event and fills them adequately to the LCEvent which is a pre-defined class in {\texttt LCIO} format. The conversion process makes use of the {\texttt LCWriter}. In order to use this converter, it is necessary to compile EUDAQ with the LCIO option enabled as the conversion will look for LCIO dependencies. 
 
\subsection{ TTree Converter} 
 
The latest addition to the EUDAQ converters list is the TTree Converter. It works in a similar way as the other converters except that the data is converted to ROOT format which does not have a pre-defined Event class like LCIO to hold the objects of \texttt{eudaq::Event}. In this converter therefore, the member variables are to be stored in the branches of a \texttt{ROOT TTree}. There are two ways in which branches can be filled during the conversion process depending on the nature of parameters which are to be stored i.e. $basic$ parameters and data $blocks$. 
 
\subsubsection{Converter Flow} 
 
The source code for the Converter can be found in \lstinline[style=cpp]{main/lib/ttree}. The dependencies are set during the compilation by providing path to the ROOT software. The Converter consists of a few components described in more detail below. 
  
\paragraph{FileWriter}
This processor prepares the output file, write the filled \lstinline[style=cpp]{TTree} to it, and books the pre-defined branches for $basic$ variables of the Event. The first part of the code given below registers the TTreeFileWriter to the core of EUDAQ. This essentially tells the EUDAQ which converter to execute given the extension of output file in the execution command. 

\begin{listing}[mybash]
    namespace{auto dummy01 = Factory<FileWriter>::Register<TTreeFileWriter, std::string&>(cstr2hash("root"));
\end{listing}

The constructor function \lstinline[style=cpp]{TTreeFileWriter()} prepares the output file and also declared branches for fundamental variables. This declaration needs special care for the type of variables in the event. The file is written in the deconstructor function.

\paragraph{EventConverter} This processor is called by the FileWriter. It is responsible for obtaining the $basic$ parameters from the raw Event and filling them to the tree as branches. As the number and type of these parameters is known already, they are stored to the pre-defined branches declared in the FileWriter. It also looks for sub-events. In case a sub-event is found, it calls the adequate process to deal with it. 
\paragraph{RawEvent2TTreeConverter} This is the process to take care of the sub-events inside an event. It will identify the type of the sub-event with a given tag, and will call the relevant converter to deal with it. There can be multiple types and each can be converted using the most suitable scheme. For example if the subtype of the event in \lstinline[style=cpp]{raw} data is \lstinline[style=cpp]{"Ex0Raw"}, it will call the \lstinline[style=cpp]{Ex0RawEvent2TTreeEventConverter}. 

\subsubsection{Ex0RawEvent2TTreeEventConverter}
The above mentioned three components are sufficient for converting the basic parameters. For conversion of the data blocks a dedicated conversion process needs to be defined. Since the type and detail of this data is only known to the user, this part is mainly to be developed by the users to meet their needs. A working example is provided with the name of \lstinline[style=cpp]{Ex0RawEvent2TTreeEventConverter}. The source code can be found in \\ \lstinline[style=cpp]{user/example/module/src/}. This converts the variable \lstinline[style=cpp]{m_block} containing \lstinline[style=cpp]{block_id} and respective blocks of data. It declares the type of event it is designed for via 
\begin{listing}[mybash]
static const uint32_t m_id_factory = eudaq::cstr2hash("Ex0Raw");
\end{listing}
So if an event or sub-event announces itself with the tag \lstinline[style=cpp]{"Ex0Raw"}, it will be sent to this converter for adequate conversion. 
\lstinputlisting[label=ls:ex0raw2std, style=cpp]{../../user/example/module/src/Ex0RawEvent2TTreeEventConverter.cc}

In this example, the content of data blocks is a vector of integers while \lstinline[style=cpp]{block_id}s are also integers so they are simply stored to respective branches without any further treatment. One of the important things to consider at this point is that the number of blocks cannot be known beforehand and they may very event to event. Therefore it will not be possible to book the \lstinline[style=cpp]{TTree} branches in advance. Hence these branches are created on the go as the blocks are identified in the \lstinline[style=cpp]{raw} event. The converter checks if the branch is already there for a specific \lstinline[style=cpp]{block_id} and fills it. If the branch is not there, it creates it. For more complex type of data blocks, an optimized approach can be adopted to store the member variables in a list of branches, as leaves of a branch etc, according to suitability for the hardware data. 

\subsubsection{Execution and Storage}

 The converter can be executed using the command line as for the other converters. The command to be executed is following:
\begin{listing}[mybash]
./euCliConverter -i input.raw -o output.root
\end{listing}
 The output file name extension tells the EUDAQ to call the adequate conversion process. The converted files are stored to the disk. This behaviour can be modified to route the converter output data towards other linked processes such as monitoring. 

\subsubsection{An Application} 
As a demonstration of the Converter at work, data from AHCAL beam test has been taken for test. The data consists of $basic$ parameters as described above, as well as sub-events tagged as \lstinline[style=cpp]{CaliceObject} and \lstinline[style=cpp]{DesyTableRaw}. Therefore two dedicated converters are written for these. The converter announces itself to the eudaq core in the following way, declaraing the type of sub-event it is designed to handle.  

\begin{listing}[mybash]
static const uint32_t m_id_factory = eudaq::cstr2hash("CaliceObject");
\end{listing}

The sub-events will then be dealt with using the conversion rules in this converter. The converter can be modified to handle the specific structure and type of the data. Different numbers of data blocks can be obtained and stored accordingly. The \lstinline[style=cpp]{CaliceObject} here, for example, consists mostly of 8 blocks though there can occasionally be up to 10 due to multiple ASIC triggers in the same event. These blocks are of varying size and contain different types of data. The converter can take the contents of block and store them in a most suitable way for the user. The blocks of zero size are not converted. The larger blocks containing the ASIC data will need the specific decoder to decipher the data, before storing in one of the allowed formats for a \lstinline[style=cpp]{TTRee} Branch. 
